Skip to content

Update Elasticsearch exporter documentation #10586

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Update Elasticsearch exporter documentation #10586

wants to merge 10 commits into from
+48 −5

Conversation

inge4pres
Copy link
Contributor

Clarified configuration options for sending queue and batcher settings in the Elasticsearch exporter documentation.

Closes #10584

What does this PR do?

Why is it important?

We are not specifying correctly how batching should be configured in each stack release of EDOT.

Checklist

  • I have read and understood the pull request guidelines of this project.
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

Disruptive User Impact

How to test this PR locally

Related issues

Questions to ask yourself

  • How are we going to support this in production?
  • How are we going to measure its adoption?
  • How are we going to debug this?
  • What are the metrics I should take care of?
  • ...

@inge4pres
Update Elasticsearch exporter documentation
f32c745 
Clarified configuration options for sending queue and batcher settings in the Elasticsearch exporter documentation.

Closes #10584
@inge4pres inge4pres requested a review from a team as a code owner October 15, 2025 13:22
@mergify Mergify
Copy link
Contributor

mergify bot commented Oct 15, 2025

This pull request does not have a backport label. Could you fix it @inge4pres? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

inge4pres
inge4pres commented Oct 15, 2025
View reviewed changes
@inge4pres inge4pres requested a review from theletterf October 15, 2025 13:24
@inge4pres inge4pres added documentation Improvements or additions to documentation docs Team:Docs Label for the Observability docs team labels Oct 15, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-docs (Team:Docs)

carsonip
carsonip reviewed Oct 15, 2025
View reviewed changes
docs/reference/edot-collector/components/elasticsearchexporter.md Outdated
stack: 9.0 9.1
```

The sending queue can be enabled and configured with the `batcher` section, using [common `batcher` settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/internal/queue_sender.go).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not technically correct. In previous versions, sending_queue and batcher are both supported, but they are separate configs configuring the queue and batch correspondingly.

Copy link
Contributor Author

@inge4pres inge4pres Oct 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok makes sense thanks - I thought sending_queue was a replacement for batcher, but no it's another setting on top.

How would you rephrase it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this paragraph (and the header) should focus on batcher and state that to keep the interaction async after enabling batcher, one should enable sending queue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks I only have one doubt: sending_queue does not seem to be aviable before v0.132.0.
if that's correct, we should not include a sending_queue section for 9.0 and 9.1.

I was thinking to structure the document with separate sections about sending queue and batching, but only if sending queue is available in 9.0 and 9.1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm I just re-read the docs in the upstream repo, sending_queue was supported in older versions too.
I'll update with dedicated sections

@carsonip carsonip requested a review from lahsivjar October 15, 2025 13:27
carsonip
carsonip reviewed Oct 15, 2025
View reviewed changes
carsonip
carsonip reviewed Oct 15, 2025
View reviewed changes
docs/reference/edot-collector/components/elasticsearchexporter.md Outdated
stack: 9.0 9.1
```

The sending queue can be enabled and configured with the `batcher` section, using [common `batcher` settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/internal/queue_sender.go).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this paragraph (and the header) should focus on batcher and state that to keep the interaction async after enabling batcher, one should enable sending queue.

docs/reference/edot-collector/components/elasticsearchexporter.md Outdated
stack: 9.2
```
The Elasticsearch exporter supports the `sending_queue` setting, which supports both queueing and batching.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first sentence kind of implies that sending queue was not supported in the past, which is not true. The key change in 9.2 (i'm not sure off the top of my head) is that we also support sending_queue::batch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, makes more sens after reading the previous comments too.
Let me rephrase it.
I wonder: if this distinction is not so clear from the upstream repo docs, should we also amend there?

@inge4pres @theletterf
fix applies_to syntax
4dc3a02 
Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 15, 2025 •
edited
Loading

🔍 Preview links for changed docs

@inge4pres inge4pres requested a review from carsonip October 15, 2025 20:06
@inge4pres
Copy link
Contributor Author

With last commit i separated the sending_queue and batching sections completely, after learning what sending_queue is via the OTel docs.

@carsonip @lahsivjar please let me know if a corresponding change should be in the upstream docs 🙏🏼

@inge4pres
Copy link
Contributor Author

@theletterf please kick in with the tabs magic if you think the applies_to are too clumsy

@theletterf theletterf added skip-changelog backport-9.0 Automated backport to the 9.0 branch backport-9.1 Automated backport to the 9.1 branch backport-9.2 Automated backport to the 9.2 branch and removed backport-skip labels Oct 16, 2025
carsonip
carsonip reviewed Oct 16, 2025
View reviewed changes
Copy link
Member

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm refraining from nitpicking too much. With this PR it should be a net improvement over the existing docs.

@carsonip
Copy link
Member

carsonip commented Oct 16, 2025 •
edited
Loading

@theletterf is it the best practice to backport doc changes? I thought docsv3 are only published from main.

@theletterf
Copy link
Contributor

@carsonip That's right, but we recently had some issues with the release notes process if docs weren't synced. +CC @colleenmcginnis do you know if we need to continue backporting docs changes for the time being?

lahsivjar
lahsivjar previously approved these changes Oct 16, 2025
View reviewed changes
Copy link
Contributor

@lahsivjar lahsivjar left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM (other than the comment by Carson on sending_queue::batch::enabled! For future/next release, we have enabled sending queue by default in ES exporter from v0.138.0.

@inge4pres
Copy link
Contributor Author

For future/next release, we have enabled sending queue by default in ES exporter from v0.138.0.

Oh so when we get an upgrade in Elastic Agent for the component, we have to update the docs again?!?
Is there a better way of handling this?
Ideally, the doc should be in a single place (upstream) and then we only add vendor-specific details for EDOT.

@lahsivjar
Copy link
Contributor

lahsivjar commented Oct 17, 2025 •
edited
Loading

Oh so when we get an upgrade in Elastic Agent for the component, we have to update the docs again?!?

Yes 😢

Ideally, the doc should be in a single place (upstream) and then we only add vendor-specific details for EDOT.

I agree 💯 . We could still do this and open PRs to upstream docs if they are not sufficient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lahsivjar lahsivjar lahsivjar approved these changes

@theletterf theletterf theletterf approved these changes

@carsonip carsonip carsonip approved these changes

Assignees

@inge4pres inge4pres

Labels

backport-9.0 Automated backport to the 9.0 branch backport-9.1 Automated backport to the 9.1 branch backport-9.2 Automated backport to the 9.2 branch docs documentation Improvements or additions to documentation skip-changelog Team:Docs Label for the Observability docs team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Docs: elasticsearchexporter batcher config vs sending_queue.batch

5 participants

@inge4pres @elasticmachine @carsonip @theletterf @lahsivjar